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THE IMPACT OF COMPUTER ASSISTED INSTRUCTION 
ON SEVENTH-GRADE STUDENTS’ 
MATHEMATICS ACHIEVEMENT 


Introduction 

The perceived problem of low mathematics achievement is a con- 
cern to education leaders at all levels of PK-16 education. Reflecting this 
problem, the Third International Mathematics and Science Study-Repeat 
(TIMSS-R) showed the weaknesses of mathematics in the U.S. compared 
to other industrialized countries when eighth grade U.S. students per- 
formed lower than those from 14 of the 38 participating nations (NCES, 
2000). In addition, 15-year-olds from the U.S. ranked between 16th and 
23rd out of 3 1 countries that participated in the 2000 administration of the 
Programme for International Student Assessment (PISA) (OECD, 2004). 
On the national level, the 2005 administration of the National Assessment 
of Education Progress (NAEP) mathematics test indicated only 36% of 
Grade 4 students scored “at or above proficient” and only 30% of Grade 8 
students scored “at or above proficient” (NCES, 2005). These results raise 
concerns about the mathematics learning of U.S. middle school students. 

Education leaders search for interventions to address issues relat- 
ed to improving mathematics achievement. This article presents findings 
from a middle school mathematics intervention implemented to improve 
students’ mathematics performance. The purpose of this empirical study 
was to determine if there was a measurable difference in achievement on 
the mathematics section of the TerraNova Full Battery standardized test 
by a sample of seventh-grade students whose teachers taught them to use 
mathematics websites and presentation software as tools to practice basic 
mathematics skills (e.g., recall, comprehension, and application; Bloom, 
1956) related to their curriculum compared to students whose teachers did 
not teach the use of such tools. 

Literature Review 

Computer Assisted Instruction (CAI) provides one possible avenue 
for education leaders to overcome or address the problem of low achieve- 
ment in mathematics. The following review of literature presents an over- 
view of the influence and impact of CAI on mathematics achievement based 
on empirical studies and meta-analyses conducted during the past 20 years. 

Positive Results of Computer Assisted Instruction 

A number of studies have suggested that the computer provides an 
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effective vehicle for improving student achievement (Bahr & Rieth, 1989; 
Bangert-Drowns, 1985; Capper & Copple, 1985). Hawley, Fletcher, and 
Piele (1986) observed that the overall mathematics achievement of third 
and fifth grade students who used CAI was higher than their peers who did 
not use computers to practice mathematics. Bahr and Rieth (1989) identi- 
fied CAI as a factor for improved mathematics achievement of disabled ju- 
nior- and senior-high students. Additional meta-analyses conducted during 
the 1 990s found positive infiuences for some aspects of CAI, such as drill 
and practice of mathematical processes (Christmann & Badgett, 1997; Siv- 
in-Kachala, 1998). Waxman, Connell, and Gray (2002) conducted a meta- 
analysis of 13 quantitative CAI studies published in peer-reviewed journals 
between 1 997-2002 and found a positive average effect size {d) (Cohen, 
1988) for CAI of .42. Social scientists consider an effect size of .2 small, 
while sizes in the range of .2 < c/ < .8 are considered moderate, and those 
greater than .8 are considered large. Traynor (2003) found CAI improved 
mathematics achievement of regular education, special education, and lim- 
ited English proficient middle school students (n = 161) on mathematics 
pretest-posttest when compared to traditional, teacher-directed practice 
techniques. The students he studied comprised intact groups based on the 
ways that the middle school scheduled students into exploratory classes. 

Mixed Results of Computer Assisted Instruction 

While some studies during the past 20 years have suggested posi- 
tive results from CAI, other studies have raised questions about its effi- 
cacy. The results of CAI remain mixed. Levin (1986) cautioned against 
an irrational exuberance toward the infiuence of CAI on achievement and 
stated that education leaders should not accept, uncritically, the claims 
of efficiency and effectiveness. Campbell, Peck, Horn, and Leigh (1987) 
found no significant difference in the mathematics achievement of third 
grade students who used CAI drill and practice activities compared to stu- 
dents who used only printed drill and practice materials. Kelman (1989) 
found drill and practice CAI activities to be dominated by monotonous 
isolated lessons, convergent questions, and narrow behaviorist activities. 
Griswold (1984) stated that CAI activities failed to foster positive student 
attitudes toward school or mathematics. Rosenberg (1991) issued a nega- 
tive review of the infiuence of computers on instruction and achievement. 
He stated that the computer failed to deliver on the promises of increased 
efficiency and effectiveness. The structure of the CAI drill and practice 
activities he found ranged from laissez-faire to heavily programmed, the 
latter featuring “teacher-proof’ lessons, intense control of student behav- 
ior, and a myopic focus on monitoring student performance. Baker, Ger- 
sten, and Lee (2002) conducted a synthesis of studies on the infiuence of 
CAI on mathematics achievement of low-achieving students. They found 
the low-achievers did not perform significantly better, and they observed 
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an average effect size of .01 (Cohen, 1988). Plano (2004) found that CAI 
activities for algebra had a non-significant predictive infiuence on student 
achievement overall but had a slightly significant infiuence on the Algebra 
achievement of English language learners. Cole and Griffin (1987) stat- 
ed that isolated CAI activities violate communication theory because stu- 
dents do not interact with each other nor with the teacher and the activities 
actually impede learning because of their convergent nature. 

Active Learning 

Like CAI, active learning is designed to improve student achieve- 
ment. Cooperstein and Kocevar-Weidinger (2004) noted that active learn- 
ing occurs when (a) the learner can construct his or her own meaning; 
(b) current learning is developed on previous learning; (c) the learner is 
involved in meaningful social interaction; and (d) the learning is built 
through the use of authentic involvement with the learning materials. 

A number of studies demonstrated that active learning is an effec- 
tive method of enhancing students’ learning in a variety of areas. Hetland 
(2000) concluded that students’ active involvement in music had an effect 
on the development of their spatial thinking. Wilson, Flanagan, Gurke- 
witz, and Skrip (2006) found that students’ active involvement in origami 
resulted in increased problem-solving ability. Cerezo (2004) discovered 
that active involvement in problem solving enhanced the learning of math- 
ematics. Huffaker and Calvert (2004) determined that active learning was 
particularly useful when used in problem solving with computers. 

The Problem 

Although some controversy exists about the effective use of CAI, 
particularly with respect to the drill and practice forms associated with 
simple knowledge development, the studies reviewed confirm a fairly pos- 
itive effect for active learning. The effect on learning occurred primar- 
ily with more complex ki nds of learning, such as open-ended, divergent 
problem solving. From the research reviewed, it is not clear, however, 
whether using active learning with simpler processes, such as those asso- 
ciated with computation-based drill and practice computer software and 
programs, will also have a positive impact on student achievement. Final- 
ly, most other computer related active learning studies use fairly complex 
components in their investigation, such as CAI combined with a prob- 
lem-based learning approach used by the teacher. In this study, we exam- 
ined the effect of a drill and practice CAI in combination with a less com- 
plex active learning follow-up exercise — namely, direct instruction of how 
to use computer presentation software to communicate understanding of 
the drill and practice exercises — on student achievement of mathematics 
skills and knowledge. 
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Methodology 

We used a pretest/posttest quasi-experimental design because stu- 
dents comprised intact groups and thus random assignment of students was 
not possible. We assigned teachers randomly to experimental and control 
groups. 

Hypothesis 

The null hypothesis: There will be no difference in achievement 
on the mathematics section of the Terra Nova Full Battery standardized 
test between students taught to use websites and presentation software as 
tools to practice basic mathematics skills and students who did not receive 
such instruction. 

Sample 


This 2004/05 study utilized a sample of seventh-grade students 
and teachers from a population of four middle school classrooms from a 
school in central New Jersey. The treatment (X) group consisted of h = 
126 and the control (O) group of « = 141 students. Only students who took 
the Terra Nova (CTB-McGraw Flill, 2003) mathematics test the previous 
year (in sixth grade) were eligible to be included in the groups. The final 
sample included all students who met the following criteria: (a) received 
a valid score on the mathematics section of the TerraNova test in the sixth 
and seventh grades, (b) were enrolled in the school for the entire sixth- and 
seventh-grade years, (c) were enrolled in the regular education program 
during the sixth- and seventh-grade years. 

The final sample included students from various ethnic groups. 
The demographic characteristics of the groups were relatively balanced in 
terms of percent of Caucasian and non-Caucasian students (see Table 1). 

Table 1 


Ethnic Characteristics (n & %) of the Seventh Grade Regular Education 
Students in the X and O Groups 


Ethnic/racial group 

O (control) 

X (treatment) 

Caucasian 

74 (58.7%) 

74 (52.4%) 

Black/African American 

30(23.8%) 

48 (34.0%) 

Hispanic/Latino 

19(15.0%) 

15(10.6%) 

Asian/Pacific 

3 ( 2.3%) 

4( 2.8%) 

Total 

126 (99.8%)“ 

141 (99.8%)“ 


Note. Sample includes students receiving basic skills instruction. 
“Totals do not add up to 100% due to rounding off. 
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The percentage of students eligible to receive free or reduced 
lunch was similar in the X and O groups. The X group contained fewer 
students who were eligible for basic skills instruction in mathematics and/ 
or language arts (see Table 2). 

Table 2 


Socio-Economic and Academic Descriptive Statistics as Number and 
Percent of Total Population for the Seventh Grade Regular Education 
Students in the X and O Groups 


Characteristic 

O (control) 
{n = 141) 

X (treatment) 
(n = 126) 

Free/reduced lunch 

44(31.2%) 

35 (27.7%) 

Basic skills/math 

42 (29.8%) 

22(17.4%) 

Basic skills/reading 

46 (32.6%) 

18(14.3%) 


Pre-intervention mathematics achievement of X-group students 
on the sixth-grade, full battery TerraNova mathematics test was signifi- 
cantly ip < .05) lower than the achievement of 0-group students. In short, 
the previous achievement of groups was not similar (see Table 3). 

Table 3 


Pre-Intervention Achievement Comparison of NCE Means for Seventh 
Grade Regular Education Students (N = 267) 


Group 

n 

Mean NCE 

SD 

p value 

t-score 

O 

126 

64.57 

15.96 

.000 

6.12 

X 

141 

53.73 

12.94 




Note. Sample includes students receiving basic skills instruction. O = control group; X = 
experimental group. 


Treatment 

We randomly assigned teachers to X (n = 2) and O {n = 2) groups 
prior to the start of the study. The teachers in the X group used mathemat- 
ics drill and practice websites and slide presentation software with stu- 
dents. The teachers in the control group used neither the websites nor the 
presentation software. The purpose of the CAI used by X teachers with 
X students was to provide practice with basic math skills related to the 
grade-level curriculum. After students became familiar with the CAI, the 
teachers taught them to use slide presentation software to create a digital 
“book-report” in which they were assigned to explain one aspect of math- 
ematics they learned via the CAI. Each student used the software to con- 
struct an explanation of the material he/she learned in the drill and practice 
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CAI. Upon completion of their report, students presented the information 
to their classmates. The students used the technology two-times per week 
for 20 weeks. 

The mathematics websites focused on drill and practice of compu- 
tation in operations, fractions, geometry, data analysis, and algebra based 
on the New Jersey Core Curriculum Content Standards and school’s sev- 
enth-grade mathematics curriculum. A site facilitator (district mathemat- 
ics supervisor) observed instruction to monitor fidelity of implementation. 
The site facilitator ensured that the mathematical content was consistent 
for all teachers and that those in the X group were the only ones using 
the mathematics websites in their presentations. The site facilitator con- 
ducted weekly classroom observations of the X and O teachers and re- 
viewed lesson plans weekly. Teachers in the X group facilitated student 
creation of slide shows so students could demonstrate their understanding 
of mathematics concepts such as adding and subtracting tractions with un- 
like denominators. 

Instrument 

The CTB/McGraw Hill TerraNova is a commercially prepared, 
norm-referenced, standardized achievement test for K-12 schools. The 
test is available in all 50 states and 275,000 students nationwide took part 
in the last norming process. More districts in New Jersey use the TerraNo- 
va to measure student achievement than other available standardized tests. 
The seventh-grade TerraNova mathematics portion contains content that 
addresses number sense and number relationships, computation and nu- 
merical estimation, operation concepts, measurement, geometry and spa- 
tial sense, data analysis, statistics, probability, patterns, functions, algebra, 
problem solving and reasoning, and communication. The reported internal 
consistency (Cronbach alpha) reliability coefficient for the mathematics 
portion is .92 for the seventh-grade Form 16. 

We used students’ Normal Curve Equivalent (NCE) scores from 
the full mathematics battery TerraNova, Form 16. We used the NCE scores 
because the school district did not make scale scores available. Students’ 
sixth-grade TerraNova mathematics NCE scores were used as a covariate 
and seventh-grade NCE scores were analyzed for differences between the 
treatment and control group students. 

Analysis and Findings 

Atwo-way ANCOVA was used to analyze the two group data (see 
Table 4). We used this statistic to control for both the pretest differences 
and a possible interaction due to socio-economic status (SES). Each stu- 
dent’s free/reduced lunch status was used as a measure of SES. Prior to the 
analysis an alpha level criteria was established at/? < .05. 
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Table 4 

Post-Treatment Results Using Two-Way ANCOVA 


Source 

SS 

df 

MS 

F 

Sig. 

Pretest 

63,763.267 

1 

63,763.267 

658.188 

.000 

Treatment 

285.948 

1 

285.948 

2.952 

.086 

Free lunch 

13.042 

2 

6.521 

.067 

.935 

Treatment X 
free-lunch“ 

163.090 

2 

81.545 

.842 

.432 

'Interaction effect between SES and treatment. 


The initial two-way ANCOVA suggests there was no statistieally 
signifieant interaetion between the SES level and the treatment. Beeause 
of the lack of interaction, it was felt that the variance from the interaction 
could be pooled in the error term and a one-way ANCOVA could be used 
to examine the effect of the treatment across the two groups. 

The subsequent one-way ANCOVA using the pooled variance for 
low and high SES showed the treatment to be statistically significant at 
the p < .05 level (see Table 5). 

Table 5 


Results of One-Way ANCOVA Using Pooled Variance 


Source 

SS 

df 

MS 

F 

Sig. 

Pretest 

64,686.683 

1 

64,686.683 

670.084 

.000 

Treatment 

764.543 

1 

764.543 

7.920 

.005 


A Cohen’s d (Cohen, 1988) was calculated to examine the effect 
size. The result showed an effect size of d = .\2, which suggested that, al- 
though significant, the effect size was small using Cohen’s interpretation 
scheme. 


Conclusions 

The data suggest that the treatment of CAI with an active learning 
component had a positive, although slight, effect on the X group students’ 
learning of basic mathematics skills (i.e., knowledge, comprehension, and 
application; Bloom, 1956) as tested by the TerraNova standardized math- 
ematics test. These data imply that the learning of basic computational 
skills can be somewhat enhanced through the use of web-based drill and 
practice exercises when students subsequently become involved in a pre- 
sentation process in which they are required to actively restate and explain 
what they learned. The small but significant effect also argues for more 
careful study of the impact of this treatment. 
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While the results are statistieally significant and positive, the find- 
ings raise the question of whether time spent on CAI drill and practice 
will enhance overall mathematics achievement in educationally significant 
ways. For example, given that half of the possible points on the New Jer- 
sey eighth-grade mathematics test come from questions that require open- 
ended problem solving, middle level educators in the state, and in states 
with similar tests, may consider whether CAI drill and practice activities 
deserve the amount of time given by the school in this study. 

The results of this study may prompt middle level leaders nation- 
wide to examine the amount of time students and teachers spend on CAI 
drill and practice activities in their schools compared to the amount of 
class time spent on problem-solving activities. In the end, leaders need to 
determine if CAI drill and practice are worth it. 

Future Research and Limitations 

Future research needs to be conducted to examine the effect of 
different aspects of computer and web-based learning experiences and ac- 
tive learning experiences. The effect size is partially a function of the stan- 
dard error of measure associated with large scale tests as well as the differ- 
ences (achievement and SES) between the treatment and control groups. 
Controls, like those provided by random assignment of students as well as 
teachers, might have shown a larger effect size. Better controls would al- 
low future researchers to establish comparability in advance instead of re- 
lying on the use of a residualized covariate. 

Because these results were a function of the domain of regular 
classroom practice, there is also always the concern that treatment ex- 
periences may have not been consistent across all classes or the control 
groups were not entirely separate from the treatment group. Although we 
took steps to minimize this possibility through the use of a site facilitator 
to provide oversight, there is a constant need for replication to ensure an 
externally valid result. 
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